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Abstract 

Accelerometers are now widely used to measure activity in large observational 
studies because they are easy to wear, measure acceleration at a resolution that hu¬ 
mans cannot report, and measure objectively the acceleration of the particular body 
part they are attached to. Using raw accelerometry data we propose and validate 
methods for identifying and characterizing walking in the free-living environment. 
Although existing methods can identify walking in laboratory settings, recognizing it 
in the free-living environment remains a major methodological challenge. We focus 
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on sustained harmonic walking (SHW), which we define as walking for at least 10 
seconds with low variability of step frequency. Our method is robust to within- and 
between-subject variability, measurement configuration and device rotation. The re¬ 
sulting approach is fast and allows processing of a week of raw accelerometry data 
(~150M measurements) in half an hour on an average computer. Methods are moti¬ 
vated by and applied to the free-living data obtained from the Developmental Cohort 
Study (DECOS) where 49 elderly subjects were monitored for one week (~300 GB 
of data). 

Keywords: Accelerometer; movement recognition; activity, walking quantification 
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1 Introduction 


We propose methods for estimating when and how people are walking based on high- 
frequency, high-throughput data obtained from wearable accelerometers. These methods 
produce a labeled time series at the sub-minute level indicating whether the person is 
walking or not. For walking periods the instantaneous step frequency and the vector 
magnitude are also estimated. Methods are not limited by the length of recording, type 
of device, or conhguration set-up, which is crucial in large epidemiological studies, where 
hundreds or thousands of subjects wear accelerometers for weeks at a time. Such studies 
collect data that exhibit extraordinary levels of heterogeneity due to the natural within- 
and between-person variability, as well as to measuring devices that are prone to batch 
effects, rotations and random artifacts. Heterogeneity makes walking prediction in the 
natural environment much more difficult than predicting walking in the lab (|Zhang et al.| 


2012 

Xiao et ah 

2014 Fortune et ah 

2014 

i Ahanathapillai et ah ,12014; 

Ermes et ah, [2008 

Mathie et ah 

2004 

Yoneyama et ah 

2014 

). The differences between data collected in 


the lab under strict protocols and data collected in large observational studies is, indeed, 
dramatic. This is probably why despite tens of papers that provide algorithms for walking 
prediction in the lab, there is currently no working solution for the problem that really 
matters: predicting walking when we do not know when and if it occurs. This made us 
coin the term data collected “in-the-wild” to emphasize the highly unstructured nature of 
the data obtained during, unsupervised, free-living, activity of humans. 

A major problem is that the definition of walking from the point of view of the ac¬ 
celerometer measurement is quite ambiguous. For example, the accelerometry signal has 
a fundamentally different structure for a 6 minute relaxed walk versus multiple bouts of a 
few seconds of walking in the kitchen interspersed among highly complex cooking activi¬ 
ties. Moreover, laboratory studies typically collect labeled data that is consistent with the 


6 minute walking (e.g. 400 meter walking (Chang et ah, 2004) or simply, 6 minute walk 
task ( Enrightf 2003)) Thus, in practice it is often hard to validate walking periods that 
are not consistent with the laboratory standardized definition. For these reasons we focus 
here on sustained harmonic walking (SHW), defined as walking for at least 10 seconds 
with low variability of step frequency. To illustrate the complexity of the data collected 
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Figure 1: “In-the-wild” tri-axial acceleration signal with different colors representing data 
from each axis respectively. Top plot depicts data from one day of activity, middle plot from 
one hour and the bottom plot from one minute. Dashed-line rectangles mark zoomed-in 
time periods for the plots below them respectively. 
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“in-the-wild” Figure [^displays three time series that represent the acceleration along three 
orthogonal axes of an accelerometer attached to the wrist of a person. At the most basic 
level, an accelerometer is an electro-mechanical device that measures acceleration along 
three orthogonal axes. The top panel of Figure presents exactly 24 hours of the ob¬ 
served data, where each axis data is shown in a different color. In this example, the hrst 
observation was taken at 12 AM, while the last was taken 24 hours later, also at 12 AM. 
The middle plot in Figure [^presents a particular one hour interval from 8.40 AM to 9.40 
AM (indicated in the top panel as a dashed-line rectangle). The bottom panel provides 
an even more detailed look at the data, by displaying the one minute interval marked 
as a dashed-line rectangle in the middle panel (from 8.51 AM to 8.52 AM). The signal 
was acquired at a sampling frequency of 50Hz, i.e. there are 50 consecutive observations 
within each second for each of the three time series. Therefore, in practice, the number 
of observations per subject quickly explodes. For example, a week of accelerometry data 
collected at 50 Hz results in 50 * 60 * 60 * 24 * 7 = 30, 240, 000 observations for each of 
the three axes. Thus, for even a small multi-subject observational study researchers are 
faced with datasets consisting of billions of observations. Our goal is to take data like 
the one shown in the top panel in Figure [T] and identify those periods that correspond to 
SHW. An example of SHW can be seen, in the lower panel of Figure 1, starting roughly 
around second 45. While some shorter walking periods may exist before second 45 it is 
impossible to validate such a claim. Thus, we will focus on methods that identify the 
period after second 45 as SHW. There are many scientihc reasons for focusing on SHW. 


First, it has been reported that in-lab gait speed is related to overall survival (Studenski 


et al., 2011). However, little is known about how gait speed in the lab is related to enacted 


walking “in-the-wild” or how the amount and timing of walking periods during the day 
is related to survival and other health outcomes. Second, it is currently unknown what 
are the daily patterns of walking, how they relate to overall activity intensity and whether 
they are objectively associated with human health. Third, changes in timing and quality 
of walking during the course of the day may be associated with aging or health outcomes. 
Thus, we introduce methods designed for automatic identihcation of SHW using tri-axial 
body-worn accelerometers signals acquired “in-the-wild”. In addition for walking periods. 
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we estimate the instantaneous walking frequency (IWF) (number of steps per second) and 
the vector magnitude (a measure of amplitude of the acceleration signal). More precisely, 
starting with the original tri-axial signal Xj(t) = {xnit), Xi 2 {t), Xi^it)}, where Xikit) denotes 
the measurement for subject i at time t from the axis k = 1,2, 3, we extract the vector 
hi(t) = {yi(t), Wi(t), Vi(t)} , where: 1) yi(t) is a binary variable with yi(t) = 1, if subject 
i was in SHW at time t, and 0 otherwise; 2) Wi{t) is the IWF for subject i at time t; 3) 
Vi(t) is the corresponding vector magnitude activity count. Using the extracted vector, 
bj(t), we can study the daily and hourly SHW percentages, the corresponding amplitude 
of walking accelerations, IWF variability as well as the length and timing of various bouts 
of SHW. These detailed summaries of walking can then be used in association studies with 
the demographic and clinical outcomes. 


Our work was preceded by a number of other approaches see e.g. Bai et ah (2012, 2014); 


He et ah (2014); Troiano et ah (2014); Xiao et ah (2014). The paper by Bai et ah (2012) 


introduced the idea of “movelets” to build subject-specihc basis functions for a variety of 
activities. They also provided recommendations for the conduct of large epidemiological 


studies collecting accelerometry data to predict activity types. Bai et ah (2014) introduced 
a set of metrics for human activity based on a hip-worn tri-axial accelerometer. They based 
these metrics on the activity time and the amplitude ratio of activity to the rest period. 


He et ah (2014) used movelets to analyze data from three triaxial accelerometers. Troiano 


et al. (2014) discuss the pressing need for a shift from count-based approaches for physical 


activity energy expenditure estimation to the characterization of the activities based on the 
features extracted from raw acceleration signals. They advocate a collaborative approach 
for development of analytical methods to accelerate the physical activity research. The 


recent technical report by Xiao et al. (2014) dealt with the inter-subject activity prediction. 
The authors addressed the issues of the device position variability and the normalization of 
the signal magnitude from different devices. While these studies have provided insights into 
the nature of activity type prediction, they have not been applied to highly heterogeneous 
data obtained from large epidemiological studies of activity “in-the-wild”. 

The remainder of the paper is organized as follows. In Section 2, we introduce the 
accelerometry data and summarize the statistical challenges. In Section 3, we describe in 
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detail our method of automatic walking detection. In Section 4, we apply the proposed 
method to the in lab data. In Section 5, we analyze real “in-the-wild” data and we close 
in Section 6 with conclusions and a discussion. 

2 Statistical Methods 

Identihcation of the types of activities based on the acceleration signals is still an under¬ 
developed. We concentrate on walking, since it is one of the most common yet important 
activities.We focns on SHW, which is the only type of walking that we can currently consis¬ 
tently estimate and validate in large epidemiological stndies. This may change in the fntnre 
with better wearable devices or improved algorithms, but estimating SHW is important. 

2.1 Data structure and EDA 

The motivating data were collected in the Developmental Epidemiologic Cohort Study 
(DECOS), a study of older individuals who are in good health. Data were collected at 
the University of Pittsburgh by Dr. Nancy Glynn as part of a larger stndy of activity 
in a normally aging population. The original aims of the stndy were: (1) to provide 
training sets for movement detection “in-the-wild” based on “in-the-lab” training data; 
and (2) to compare the activities performed (such as walking and sitting) “in-the-lab” 
and “in-the-wild”. We concentrate on the data from A = 49 individuals who have both 
“in-the-lab” and “in-the-wild” accelerometry measurements and associated demographic 
and clinical covariates. Each “in-the-lab” experiment consisted of a series of 31 activities 
including normal and fast walking, sedentary activities (e.g. lying still, writing and dealing 
cards), low-impact activities (e.g. shopping, folding towels and kneading) and higher- 
impact activities (e.g. vacnuming and chair stand). All these activities were performed 
in a laboratory in a presence of a hnman observer who marked the start and end of each 
activity. 

“In-the-wild” accelerometry data activities were collected over a one week period in the 
natural living environment. In contrast to “in-the-lab” experiments, the activity labels for 
“in-the-wild” data are unavailable. 
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We start by exploring some portions of the 49 million-long time series for one of the 
DECOS snbjects’ “in-the-wild” data. Fonr panels in Figure [^display examples of the data 
that we labeled (1) resting, (2) change in position, (3) active, but not periodic (compound) 
and (4) locally periodic; data are shown in the left panels. The combined data shown in 
Figure [^represent less than 0.04% of the data for this particular subject. The middle panels 
display the corresponding local means (solid lines with colors corresponding to those of the 
original times series) and standard deviations (shown as solid lines with a circle and colors 
corresponding to those of the original times series). Both the local means and standard 
deviations are based on a 10-second moving window. The right panels display the Fourier 
spectrum for each axis separately with colors corresponding to those of the raw data, 
respectively. The resting signal is characterized by a flat, very low amplitude time series 
centered around the local mean across all 3 axes (top left panel). Local averages of the 
resting signals are roughly constant, while the local standard deviation is also approximately 
constant but very close to zero relative to the size of the signal across the 60-second time 
interval (middle top panel). The Fourier spectrum displayed in the right-top panel of Figure 
1^ does not indicate any sizable periodic component, which conhrms the observed nature 
of the data. The signal for change in position, displayed in the second row/left panel of 
Figure is visually similar to the resting signal, with the exception of a change in the 
mean of two of the time series roughly between seconds 5 and 25. This change corresponds 
to an increase in standard deviation. These characteristics are well captured by the local 
mean and standard deviation of the time series displayed in the middle-second row panel 
of Figure The Fourier spectrum has a declining peak at zero, which is characteristic 
of a low frequency change, associated with the observed change in the mean. Such a 
spectrum is characteristic of no signal variability for all but the lowest frequencies. The 
compound signal, displayed in the third row/middle panel of Figure]^ is characterized by 
large changes both in the magnitude and frequency of the signal along all 3 axes. The mean 
and standard deviation exhibit substantially larger time-dependent changes compared to 
the mean and standard deviations of the other three types of signals displayed in Figure]^ 
The Fourier spectrum of the compound signal (third row/right panel of Figure]^ indicates 
that the signal is a combination of many fundamental frequencies that seem to not be 
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well synchronized across the 3 axes (the peaks and valleys of the three Fourier spectra 
do not overlap). The locally periodic signal, displayed in the fourth row/right panel of 
Figure]^ is characterized by a more systematic, periodic-looking time series around quasi¬ 
constant local means. This latter characteristic of the data can be seen in the local means 
displayed in the fourth row/middle panel of Figure]^ Indeed, the local averages are almost 
indistinguishable from the local averages displayed in the corresponding panel for resting. 
This indicates that resting was probably “resting standing”, as the average orientation of 
the device is roughly the same as for walking. A different posture would result in a flat, 
but dramatically different local mean level. The local standard deviations are larger for 
the locally periodic signal than for the resting signal, as accelerations associated with the 
periodic movement vary a lot more along their local means. The Fourier spectrum for 
the locally periodic signal, displayed in the fourth row/right panel of Figure]^ shows that 
there is a small number of frequencies that explain the observed variability of the signal. 
Moreover, the spectra of signals corresponding to different axes are better synchronized 
than spectra for the compound signal (compare panels on the right in the third and fourth 
row of Figure [^. We will use this property of the local periodic signal to detect walking. 
Recognizing walking in complex data obtained from observational studies “in-the-wild” 
is difficult because: 1) the timing and length of walking periods are unknown; 2) the 
intensity and frequency of walking can vary substantially within-subject and dramatically 
between subjects; 3) compound movements can include walking and may correspond to a 
large number of heterogeneous behaviors; 4) orientation, location, and type of device can 
substantially alter the type of signal along individual axes; and 5) Fourier decompositions 
become noisier in shorter intervals (3-5 seconds). 

2.2 Modeling approaches 

In this section, we provide the intuitive basis of the proposed accelerometry signal modeling 
approach. A fundamental property of SHW is that it consists of a number of steps, which 
are executed within a limited range of frequencies. Our approach will take advantage of 
this property and estimate short periods that exhibit strong periodicity around a range of 
possible frequencies. Frequency of movement will be allowed to continuously adapt to a 
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Figure 2: Panels represent four different types of signal with the units on the y-axis in 
earth acceleration {g = 9.81m/s^) and time in seconds on the x-axis for the 3 orthogonal 
axes represented by different colors. Specihcally, First row, left panel shows a portion of a 
resting signal, Second row, left panel shows a part of the change in position signal, Third 
row, left panel shows compound signal and Fourth row, left panel shows a locally periodic 
signal. Middle column displays local averages and standard deviations of the signal for the 
respective types of the signal and the right column displays Fourier spectra calculated for 
60 second time interval. 
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range of walking motions both within- and between-subjects. Methods are based on Fast 
Fourier Transform (FFT) along each of the three axes. 

The use of local FFT may be perceived as not being particularly new, though the 
implementation is designed and highly modihed to address the size and complexity of the 
problem. Indeed, for experts in accelerometry data it should be obvious that a brute force 
approach may result in serious biases in estimating the timing and duration of walking 
and may not generalize well to a population of subjects. We also propose an innovative 
approach to using the information in the spectrum, by incorporating information at multiple 
of walking frequencies. 

2.3 Statistical challenges 

A number of issues arise in modeling accelerometry data generated during SHW. First, 
walking is highly variable across people. Second, the device can shift or rotate relative 
to its original location during walking. This requires careful treatment of the signal, as 
such changes happen at unknown intervals and for unknown durations. The local Fourier 
transform is well suited for addressing those problems, as the associated changes in the 
accelerometry signal are typically associated with low frequency changes (see, for example, 
Fourier spectra for the change in position data in |^. Thus, low frequency components 
can be hltered out using a high-pass hlter (essentially subtracting the Fourier coefficients 
corresponding to low frequencies). 


3 Method for automatic walking detection 


3.1 Definitions and notation 


We have dehned SHW as accelerometer activity signals that are performed for at least 
10 seconds and are periodic with a roughly constant walking frequency. Here we propose 
a method for automatic recognition of SHW intervals together with estimation of the 
periodicity of the dominating component. The reported frequency of human walking “in- 
the-wild” ranges between 1.4 - 2.5 Hz ( Pachi and Ji| 2005). However, we will conservatively 
focus on the range 1.2 - 4.0 Hz to include slow walking of older individuals and running. 
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We assume that any activity where the dominant frequency of movement is in this range 
is SHW. 

Let the measured signal be x(t) = {xi{t),X2{t),x^{t)}, where Xkit) denotes the mea¬ 
surement at time index t along the axis k = 1,2,3. For notational simplicity we drop the 
subject index. Time is indexed from 1, which refers to the hrst accelerometry observation 
and not the wall clock. Increments are in fractions of a second. For example, time t = 151 
corresponds to the beginning of the third second of monitoring when data are collected at 
50 Hz (50 times per second). 

We denote by /o the sampling frequency of the data expressed as number of observations 
per second. For example, if the acceleration signal is sampled at the /o=50Hz frequency 
data consist of 50 samples per second for each axis for a total of 150 observations per second 
for the three axes. 

Next, we formally dehne the vector magnitude at time t as 

vm(t) = ^/xi(ty + X2(ty + xsity. (i) 

However, in order to compensate for the constant earth gravity we subtract 1 and dehne 
the vector magnitude count for a given period [f — r/2, t 4- r/2] as, 

^ [i+^/2] 

Vr(t) = - \vm(u) — 1 | , ( 2 ) 

T ^^ 

U=[t—T/2] 

where r is the window size expressed as number of sampling points. Our main goal is to esti¬ 
mate the vector y(t), where y(t) = 1 corresponds to SHW and y(t) = 0 corresponds to non- 
SHW. This is achieved by employing a short-time Fourier transform (STFT) for each axis 
separately. The discrete Fourier transform (DFT) for axis k is Xk{f) = Ylt=o , 

where / is the frequency index, t is the time index and T is the total number of obser¬ 
vations. We dehne the short time Fourier transform (STFT) at time t for axis k of the 
acceleration signal x(f) as Xk{t,f]T) = ^ where r is a tuning 

parameter specifying the number of observations in the interval centered at t. The weights 
h{u) are used to assign more weight to observations that are close to t. We use the Hann 
window which is dehned as, h{u]T) = 0.5[1 — cos{27rM/(r — 1)}]. The Hann window is a 
popular choice in applied time series analysis because it has been shown to reduce aliasing 
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(Harris, 1978). For this reason it has been used extensively in multiple applications includ¬ 


ing speech processing (Benesty et al.| 2012), seismology (Van Herwijnen and Schweizer 


2011), vibro-acoustics (Antoni, 2005) and sleep analysis (Crainiceanu et ah, 2011). The 


spectrum is dehned as S'fc(t,/;r) = /;r)|, where | ■ | denotes the absolute value of a 

complex number. A first step towards the automatic walking recognition is to introduce a 
function that “combs” the spectrum for those frequencies that are likely to be related to 
walking. A comb function is dehned by a fundamental frequency, say s, and the thickness 
of the comb “teeth”. All teeth are considered to be of the same width (same number of 
frequency bins) and the total number of teeth is rim- For any frequency s, we dehne a 
neighborhood, Ng = {s — s, s where T is the length of the window expressed in 

seconds, where the local FFT is applied. Thus, Ng is the shortest frequency interval cen¬ 
tered at s and consisting of three frequencies. Intuitively, we are using “a comb with very 
narrow teeth”. The comb family of functions, C'(/;s), indexed by s, is dehned as 


CU;s) 


1 for / e 
0 


(3) 


While notation is a bit involved, the intuition behind the comb functions is straightfor¬ 
ward. Consider the spectra shown in the top panel of hgure that correspond to a par¬ 
ticular walking interval. The bottom panel displays the comb function corresponding to 
the fundamental frequency s = 2.067 Hz, the peak of all three spectra in the top panel 
(black teeth). The tooth centered at 2.067Hz also contains the frequencies 1.967Hz=2.067- 
1/lOHz and 2.167Hz=2.067-l-l/10Hz, because we are using a window of size T=10s and the 
resulting spectral frequency is equal to 1/T = O.lFfz. There are additional “comb teeth” 
centered at 3s/2 = 3.102i7z and so on. Figure also displays another comb shown in 
red corresponding to s = 2.9)lHz. Note that integrating (summing) the spectrum in areas 
corresponding to the “black” comb will result in a higher value than for the spectrum inte¬ 
grated in the areas corresponding to the “white” comb, which does not match the spectral 
peaks. Thus, the intuition behind our proposal is relatively simple: use comb functions, 
multiply them with the spectrum, add these values and choose those combs that maximize 
the partial area under the spectrum. This is especially useful when the process is repeated 
millions of times. The comb function idea was inspired by the widely used comb hlter of 
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Figure 3: Top figure shows Fourier spectra of tri-axial acceleration signal of walking. Bot¬ 
tom figure shows “Comb” function C'(/; s) for s = w{t) = 2.067 Hz [black) and for s = 2.81 
Hz [white). Black lines correspond to the spectral lines representing walking signal com¬ 
ponents while white lines miss those spectral lines completely. 


harmonic components (Deller et ah, 2000). However, in contrast to this hlter that nses the 
entire range of spectral components, onr comb nses only freqnencies between s and UmS. 
In walking recognition it is necessary to limit the nnmber of high freqnency signals, as the 
signal is noisy, which indnces spnrious high freqnency spectral spikes. In simpler terms, 
FFT does not eliminate the noise, bnt re-distribntes it, especially in the high freqnency 
range. 
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3.2 Prediction of walking periods and its characteristics 


We now provide the technical description of the SHW prediction approach. Specihcally, for 
each axis k, we dehne the area under the full spectrum, Sk{-), as ISkit) = f Sk{f, t; r) df 
and the area under the spectrum corresponding to the comb function C{f, s) as I Skit, s) = 
If=o df. All the functions under the integral are positive and ISkit) > 

ISkit, s) for every s and t, except for degenerate cases (e.g. when all values of the spectrum 
are zero). We dehne 


s) = 


ISkit,s) 


(4) 


ISkit) - ISkit, s)' 

which is a measure of the size of the periodic content of the accelerometry signal along axis 
k corresponding to the comb function C(-; s) relative to the total variation associated with 
the signal along axis k. 

Let Yit,s) = maxk{Ykit,s)}, which is the maximum of the fraction of the signal ex¬ 
plained by the frequency s along the three axes k=l,2,3. This is the time series that is 
used to estimate SHW periods. For a given a threshold 6, we estimate that a person is in 
SHW as 


[u = 


1 if max^ eDf y it, s) > 6; 

0 , 


(5) 


for every u G \t — t/ 2,t + t/2], where 5 is a tuning parameter and Df is the family of 
frequencies corresponding to walking. Here we use Df = [1.2, 4:]Hz, which is a conservative 
choice. Once the estimation process of ysi-) is hnished for the interval [t — r/2,t -|- r/2], 
the interval is shifted by a second and the procedure is repeated. Due to the use of 
overlapping windows, for every 1-second intervals there will be 9 estimates of whether or 
not the person is walking. Based on these estimators we predict that the person is walking 
in that particular second if any of the 9 intervals estimated the person to be walking. 

For periods that are estimated as walking the IWF index will be estimated as the 
frequency, s, that maximizes H(t, s). More precisely. 


wit) 


argmax^g^i^lH(t, s)} for y^it) = 1 
NaN 


The complete description of the approach is summarized in Algorithm 


( 6 ) 
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Algorithm 1 

Input: x(t) - accelerometry signal, fs - sampling frequency, T - observation time, r - time 
window, 6 - threshold, Smin = Smax = 

Output: y{t) - walking indicator, v{t) - vector magnitude, w{t) - IWF. 

Step 1. Compute the value of the “comb” function C{f] s) for each value of s G -D/ 

Step 2. Compute the spectrum, Sk(t,f,T), for each axis k = 1,2,3 and the area under 
the spectrum I Skit) for each t 

Step 3. Compute the partial area under the spectrum I Skit, s) for each s and each k. 
Step 4. Calculate the periodic content of the signal Yk{t, s) = s) 

Step 5. Estimate ys(t) with max^g^)^ Ykit, s). 

Step 6. For the times t with ys{t) = 1 estimate w{t) by Ending the s that maximizes 
Y{t,s). 

Step 7. Identify walking if any of the estimators ys{t) is 1. 
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The properties of the proposed method depend on three tuning parameters: the short- 
time window length, r, used for the Fourier transformation, the threshold 6 used for esti¬ 
mating the cut-off point on ma.Xs^Df{y(t, s)} that indicates SHW/non-SHW and number 
of harmonics (comb teeth) used for calculation of ISk(t,s). We evaluate the tuning 


parameter selection in Section 4T and provide recommendations for their choices based on 
“in-the-lab” experiment. The properties of the proposed method are studied in Section 
1^ where ROC curves in conjunction with the corresponding areas under the ROC curves 
(AUCs) are estimated for the SHW detection in “in-the-lab” data. 


4 Validation using “in-the-lab” data 

To validate the method, we used data collected during the “in-the-lab” phase of the DECOS 
study. We use data collected on A = 47 elder participants, age (Ql=74, median=78, 
Q3=78.28), 25 males and 22 females. During the “in-the-lab” phase of the experiment all 
participants were asked to perform a series of physical tasks including: lying still, standing 
still, washing dishes, sitting still, dough kneading, dressing, folding towels, vacuuming, 
shopping, writing, dealing cards, standing up from a chair, walking for 20 meters, walking 
for 20 meters with arms crossed on the chest, fast walking for 20 meters, fast walking 
for 20 meters with arms crossed on the chest, treadmill walking at 1.5mph for 5 minutes, 
walking for 40 meters and fast walking for 400 meters. The experiment was supervised by 
a trained professional who recorded the times of the beginning and the end of each task. 
For walking prediction we focused on 400-meter walk, whereas all non-walking activities 
including dressing, shopping and standing up from a chair were classihed as non-walking. 

4.1 Selection of the tuning parameters 

To apply our method to the raw accelerometry signal, several tuning parameters need to 
be specihed: the window length, r, the threshold 6 and the number of harmonics (comb 
teeth) Hm- Here, we discuss their choices. 

The selection of the length of the moving window r, determines the resolution of the 
spectrum which is inversely proportional to the signal length; for example a value of r = 10 
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seconds corresponds to a frequency resolution of 0.1 Hz. There are trade-offs in choosing 
r as: 1) longer windows result in more precise spectrum estimation if the time series are 
relatively stationary in the time window; and 2) shorter time windows are more likely 
to capture changes in walking frequency or changes in activity type. The optimal choice 
of r depends on the unknown properties of the observed signal. Based on the empirical 
evaluation of “in-the-lab” data, we have found that a 10 second interval is long enough for 
people to maintain their IWF yet short enough to be sensitive to changes in the transition 
from SHW to non-SHW. Thus, in the remainder of this article, we use r = 10 seconds. 

Choice of the threshold 6 and the number of harmonics rim are inter-related. The 
proportion of the variability (max^g^)^ y(t, s)) explained is an increasing function of the 
number of harmonics We study 5 as a function of for = 2,..., 17, where 
the limit 17 is determined by the sampling frequency of the raw accelerometry data. We 
estimate the density function of maXsgDj^ H(t, s) for all SHW and non-SHW periods for all 
subjects. The parameter 6 was then estimated for each subject separately as the intersection 
between the subject-specihc SHW and non-SHW density functions. Figure displays the 
boxplot of the estimated 6 values across subjects as a function of the number of harmonics, 
rim- As expected, the median 6 increases as a function of rim, as every subject-specihc 
S is an increasing function of rim- While some between-subject variability exists in the 
estimation of 6 at every value of of having a population level value simplihes the 
procedure considerably. For example, when rim = 6 harmonics are used an estimated 
median value of 6 at the population level is equal to 0.115. The selection of the number of 
harmonics, rim, is important for the estimation of IWF, u{t) (equation]^. In principle, we 
would like to utilize as many frequencies as possible without degrading IWF estimation. 
Indeed, a quick inspection of Figure indicates that there is information about walking 
frequency at multiple harmonics (multiples of the fundamental frequency). If data were 
perfectly periodic then one would expect the harmonics to line up perfectly. However in 
practice, walking accelerometry data is not perfectly periodic and contains a substantial 
amount of noise. 

To study the choice of rim, we have calculated the IWF, oj(t), for every subject at each 
time point during their 400-meter walk. For every subject, we then calculated the coefficient 
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Figure 4: Boxplot representing distributions of subject-specific delta vs. rim ■ 

of variation of IWF during this period. Given that this is a well controlled experiment, the 
coefficient of variation of the IWF is expected to be small. Figure [^displays the coefficient 
of variation as a function of number of harmonics for every subject (transparent lines) 
together with their mean (dashed line) and median (solid line). We have observed that 
the median coefficient of variation is relatively stable for rim = 2,..., 6 and it starts to 
increase for larger values of rim- Thus, to use as much periodic information as possible and 
still keep the coefficient of variation small, we selected rim = 6. The road to complexity 
is paved with tuning parameters and we had to make some choices as well. Specihcally, 
we used a r = 10 second with 9 second overlap between consecutive windows, rim = 6 
harmonics for calculating the comb function, and 5 = 0.115 as the threshold for percent 
variability explained used for predicting SHW. We also explained exactly why we made 
these choices and provided the technical details. We are not aware of any published paper 
that considered the complexity of predicting SHW “in-the-wild” and with such specihc 
attention to details. 

Many other choices could be made for the tuning parameters. For example, using a 
subject-specihc threshold 6 may seem like a more powerful approach. However, with such 
a choice the algorithm would need to be retrained and only work on subjects who have 
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Figure 5: Coefficient of variation as a function of number of harmonics 71^- Transparent 
lines - subject specific, dashed line - across-subject mean, solid line - across-subject median. 

carefully collected “in-the-lab” data. Moreover, in our experience, walking characteristics 
can change dramatically within the same person within the same day. Thus, it makes sense 
to identify those thresholds that do well across a population of subjects and time. 

4.2 The gold standard for walking signal indices 

In an ideal scenario one would need the gold standard labeling of walking and non-walking 
activities. Such gold standards exist for “in-the-lab” experiments, which contain the labels 
provided by human observers. As an example. Figure [^displays the acceleration data with 
the dashed-line box indicating the portion of 400-meter-walk period identified by the human 
observer as a 400-meter walk. This plot shows that even gold-standard labels are actually 
“shifted” by about 45 seconds relative to the walking signal. Correspondingly, the period of 
walking activity seems to be about 45 seconds shorter than the actual signal visible as the 
low level of activity before the beginning of the high-amplitude, quasi-periodic signal. This 
can be due to multiple factors including imperfect synchronization of watches, time elapsed 
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Figure 6: Tri-axial acceleration signal from 400 meters walk. Dashed-line box marks time 
indices given by human observer. 

between the beginning and recording of the task, and basic observer error. We chose to 
manually inspect all “in-the-lab” data for each subject and relabel walking as periods that 
seem to more closely correspond to the SHW. We have found that the overlap between labels 
of the human observer and our improved labels was below 80% in 18 out of 49 subjects. 
For 29 out of the 49 subjects the overlap was above 80% (see Figure [^. Additionally, 2 
out of 49 subjects did not perform the 400-meter walk task. Proper relabeling is crucial 
when one is interested in quantifying the performance of walking algorithms. Indeed, an 
algorithm designed to recognize walking patterns like the ones between minutes 1 to 5 in 
Figure would have a hard time recognizing as walking the time series before minute 1. 
We contend that no algorithm should and propose to improve the human observer labels by 
direct inspection of the data. These improved walking labels were used as a gold standard; 
however, we have also calculated ROC curves and corresponding AUCs for the original 
labels provided by the human observer. The AUC was excellent (approximately 0.95) with 
a very tight distribution across subjects. In contrast, the performance relative to the copper 
standard was much lower. We contend that this, in fact, is a good thing, as we trust the 
gold standard a lot more than the copper one. Figure [^provides the boxplots for the AUC 
using the human observer (copper standard) and the visual inspection plus human observer 
(gold standard). 
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Figure 7: Summary of the overlap between times given by human observer and times 
corrected via visual inspection of the data. 
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Figure 8: Boxplots for AUCs calculated for original indices (left) and for corrected indices 
covering whole and only walking activity (right). 
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5 Analysis of “in-the-wild” data 


“In-the-wild” data were collected continuously for seven days resulting in approximately 150 
million observations for the three axes of each of the 49 subjects. We applied the approach 
described in Section to estimate when SHW occurs together with the corresponding 
IWF and vm(t). We used r = 10 and 6 = 0.115 and a range of possible IWF between 
1.2 and 4.0 steps/second. This range is much more conservative than the range reported 


in the literature (Pachi and Ji, 2005) to account for the slower walking of older adults 
(Studenski et ah, 2011[ ) as well as for running for active individuals. SHW is estimated in 
time increments equal to 1 second, so consecutive windows overlap by 90% and a one second 
interval is declared to be SHW if any of the intervals containing it was estimated it to be 
SHW. The length of each walking bout is the number of consecutive time windows where 
walking was estimated to occur plus the window length multiplied by the overlap. For 
example, if walking was detected in 8 consecutive windows, the walking bout was classihed 
as lasting for 8 + 0.9 x 10 = 17 seconds. It is important to state that we can not estimate 
a length of the walking bout that is shorter than the length of the window r. Therefore, 
if walking was detected for only one window it will be classihed as a bout lasting for 10 
seconds. 

Figure displays the total SHW time (top panel) for each of the 49 participants to¬ 
gether with the corresponding total number of walking bouts (bottom panel). Both panels 
present per-day averages obtained by averaging over the 7 days of activity for each subject. 
Participants results are sorted in decreasing order according to the estimated average total 
walking time. Color shading corresponds to different length of walking bouts. For example, 
the width of the lightest bars in the top panel of hgure rehects the total walking time 
obtained from walking bouts equal to 10 seconds. The lightest bars in the bottom panel 
of Figure display the total number of bouts equal to 10 seconds. Results indicate that 
long SHW time does not necessarily indicate a large number of SHW bouts. For example, 
subject 1 was estimated to have the longest SHW time per day (~140 minutes), spread in 
210 SHW bouts, which is about the third quartile of the number of SHW bouts for this 
population. Subject 1 had the largest number of long SHW bouts (> 60 seconds, shown 
in deep red). In contrast, subject 10 had an estimated average of 100 minutes of SHW 
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per day, that is about two thirds of the SHW time for subject 1. However, subject 10 had 
more than 300 SHW bouts, or about 50% SHW walking bouts than subject 1. Many of 
these SHW bouts correspond to short SHW walking bouts. Results also indicate that the 
majority of daily walking bouts for all subjects are between 10 and 30 seconds. Similarly, 




Figure 9: Bar plot presenting the total time of walking and corresponding number of bouts. 


Figure [redisplays the box plots of IWF for each subjects. As in the previous plots, subjects 
are sorted according to total walking time, whereas the width of boxes is proportional to 
the number of walking bouts. The variability of IWF in this population is striking. Indeed, 
most subjects seem to have a wide range of IWF. For example, subject 40 had bouts of 
walking at 2.6 and 1.2 steps per second, which hts the currently reported range of IWF 
in the population. We hnd these plots to be deeply informative and contend that they 
open entire areas of research in activity that would not have been possible using currently 
available methodology. Figure [II| provides the lasagna plot (Swihart et ah, 2010) for daily 
walking characteristic of all 49 subjects. The color intensity corresponds to the number of 
minutes of walking within a one hour window. Each row corresponds to one day and data 
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Figure 10: Box-plot presenting IWF for each participant observed “in-the-wild”. Width of 
boxes is proportional to the number of observations. 

for individual subjects are separated by dashed lines. Subjects were sorted according to 
total walking time using the same ordering as in hgure with subjects with the highest 
average daily walking time shown at the top of the lasagna plot. 

6 Summary and discussion 

We have introduced the dehnition of SHW together with the prediction algorithm. We 
have also introduced additional quantihcation methods of SHW using high-density tri- 
axial accelerometry data. We contend that this is the hrst procedure designed to deal 
with the complexity of the data collected “in-the-wild”, the actual living environment of 
individuals. The method is independent of the type of device and sampling frequency of 
the raw data, as long as the sampling frequency is large (say >30Hz.). Methods have 
been built with the idea of being generally usable and easy to re-tune based on basic 
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Figure 11: Lasagna plots presenting number of minutes of walking per one hour. Rows 
represent consecutive days. Dashed lines separate different subjects. 

principles. For example, different people develop higher or lower accelerations (higher or 
lower amplitude signals). However, the statistic used for detecting SHW re-normalizes 
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data within individuals. Indeed, equation Q indicates that decisions are based on the 
fraction of spectral power associated with walking and not on the total power. Thus, the 
normalization is intrinsic and automatic. Once SHW is identihed, the vector magnitude 
can be calculated and the amplitude of SHW is recovered. We believe that methods work 
well because they are based on the quantihcation of the “local degree of periodicity” of all 
tri-axial time series as well as on sweeping the relevant frequencies using the comb function. 
However, it should be noted that the performance depends strongly on the assumption that 
SHW is a repetitive and sustained process in a particular time window. For example, it 
could be possible for a window of 10 seconds to contain 3-4 seconds of walking to not be 
recognized as walking. Thus, our method will tend to work well at recognizing sustained 
walking for more than 10 seconds. This problem is extremely difficult and a solution for 
it exceeds the scope of our current paper. Detection and analysis of a few or even single 
steps using extensions of the proposed method will be the subject of further studies. 
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